The present invention relates to technology of coding a signal such as a video or audio signal and technology of recording coded data on a storage medium.
In coding a video signal, transform coding is usually employed. According to this technique, an image is divided into a plurality of blocks, each composed of a multiplicity of pixels neighboring each other, and an orthogonal transform, like discrete cosine transform, is performed on each of these blocks. Also, a transform coefficient is quantized based on a predetermined quantization unit (quantization parameter), and then compression-coded by a variable-length encoding technique such as Huffman coding.
Also, in coding a moving picture like a TV signal, inter-frame coding is performed by utilizing a correlation between frames. According to the inter-frame coding technique, an image at a frame to be coded is predicted using, as a reference frame, another frame preceding or succeeding the frame in question on the time axis, and a prediction error thereof is coded and transmitted or recorded. The inter-frame coding is performed on the basis of a block composed of a plurality of pixels. And the motion quantity of each block is also transmitted or recorded along with the prediction error.
FIG. 22 illustrates a coding technique according to the MPEG-2 (Moving Picture Experts Group) standard, which is an international standard of moving picture coding. According to the MPEG-2 standard, intra-frame coding, inter-frame forward predictive coding and inter-frame bidirectional coding are performed. A frame, on which intra-frame coding is performed, is called an xe2x80x9cI-framexe2x80x9d. A frame, on which inter-frame forward predictive coding is performed, is called a xe2x80x9cP-framexe2x80x9d. And a frame, on which inter-frame bidirectional coding is performed, is called a xe2x80x9cB-framexe2x80x9d. In a GOP (group of pictures) consisting of a plurality of frames, at least one of the frames is intra-frame-coded.
According to such a coding technique, variable-length encoding is employed, and therefore, the number of bits generated cannot be recognized accurately until the coding processing is over. Thus, feedback control is performed such that a cumulative error, which is an accumulation of errors between numbers of bits generated and target numbers of bits, becomes zero and that an average coding rate during a predetermined interval can be approximated at a target coding rate.
Also, a variable-rate coding technique for coding a video at a coding rate variable with the complexity of coding an individual scene thereof is lately applied to a DVD (digital video disc) and so on. According to this technique, a coding rate is increased for a scene with a high complexity of coding, thereby reducing coding noise. On the other hand, as to a scene with a low complexity of coding (such as a scene with little motion), the coding rate is decreased, thereby balancing the overall image quality of the video and reducing an average bit rate.
Variable-rate coding is implementable most easily by fixing a quantization parameter for quantization processing. If the quantization parameter is fixed, then the resolution of quantization is also fixed, but the coding rate for a scene with a high complexity of coding gets higher than a scene with a low complexity of coding. Accordingly, the coding rate becomes variable.
FIG. 23 is a block diagram illustrating a configuration for a conventional coder for controlling an average rate by a feedback control technique. As shown in FIG. 23, an input video signal is quantized by a quantizer 701 using a quantization parameter Q, which has been determined by a rate-controlling-parameter determiner 704. The output of the quantizer 701 is variable-length encoded by a variable-length encoder 702 and then output as coded data. A bit number calculator 703 calculates a number of bits B of the coded data generated per unit time t.
In the rate-controlling-parameter determiner 704, an error calculator 711 obtains an error d (=Bxe2x88x92BT) by subtracting a target number of bits BT, which is derived from a target coding rate, from an actual number of bits generated B. A cumulative error calculator 712 accumulates the errors d, thereby obtaining a cumulative error D. A quantization parameter determiner 713 controls the quantization parameter Q such that the cumulative error D becomes zero.
If the cumulative error D is a positive large value, then the quantization parameter Q is increased to reduce the number of bits generated B. Conversely, if the cumulative error D is negative, then the quantization parameter Q is decreased to increase the number of bits generated B. The quantization parameter Q may be given by the following equation:
Q=Qxe2x80x2/(1xe2x88x92D/T)
where Qxe2x80x2 is a quantization parameter used in previous quantization processing and T is a time constant used for the feedback control. During the interval T, the cumulative error D becomes zero and the average coding rate during the interval T can be controlled at the target coding rate.
In this case, by setting the time constant T at a sufficiently large value, variable-rate coding can be performed during the interval T. This is because if the time constant T is large enough, the quantization parameter Q hardly varies and is converged at a substantially constant value. As a result, a coding rate increases for a scene with a relatively high complexity of coding, but decreases for a scene with a relatively low complexity of coding. That is to say, coding processing can be performed at a rate variable with the complexity of coding an individual video scene.
The conventional coding processing with average rate control, however, has the following drawbacks.
The sensitivity of the human eyes to coding noise resulting from coding processing greatly differs depending on the complexity of coding an individual video scene. Specifically, coding noise generated in a scene with a low complexity of coding (e.g., a substantially uniform picture with almost no motion such as xe2x80x9cblue skyxe2x80x9d) is more perceivable to the human eyes compared to a scene with a high complexity of coding (e.g., a complicated picture with a lot of motion such as xe2x80x9ccrowdxe2x80x9d). In other words, the human vision is more sensitive to coding noise generated in a scene with a low complexity of coding than that of a scene with a high complexity of coding.
Thus, in coding a video with a substantially constant quantization parameter, even if a coding noise generated does not seriously affect the image quality of a scene with a high complexity of coding, non-negligible deterioration may be caused by the same noise in a scene with a low complexity of coding.
Also, in controlling an average coding rate according to a conventional technique, if a cumulative error is large when a scene with a high complexity of coding is changed into a scene with a low complexity of coding, the quantization parameter is controlled at a larger value. In a scene with a low complexity of coding, however, even if the quantization parameter is increased, the number of bits generated does not decrease so much. Thus, although the average coding rate can be controlled at a value closer to the target coding rate, deterioration in image quality might surpass the advantageous effects attained by the rate control.
Furthermore, the feedback control might cause oscillation in quantization parameter or number of bits generated, and the image quality might considerably change in such a case.
In general, optimum number of bits and quantization parameter may be determined for an individual scene by estimating the complexity of coding a signal in advance. In such a case, however, since an additional time is needed to estimate the complexity of coding, it is difficult to perform coding processing in real time.
An object of the present invention is to reproduce a signal of quality at a low coding rate even when a signal continuously supplied should be coded in real time.
Specifically, a method according to the present invention is adapted to code a signal, representing a video or being accompanied with a video, by quantization processing. The method includes the steps of: detecting a scene change in the video; if the scene change has been detected, determining a quantization parameter adaptively to a scene newly appearing after the scenes have been changed; or unless the scene change is detected, determining the quantization parameter at such a value as to stabilize a coding rate; and performing the quantization processing using the quantization parameter determined.
According to the present invention, the quantization parameter is controlled at such a value as to stabilize a coding rate within the same scene. On the other hand, if scenes have been changed, the quantization parameter is determined adaptively to the newly appearing scene. Thus, even if a continuously input signal should be coded in real time, coding processing can be performed adaptively to the individual scenes, while controlling an average coding rate at the same time. As a result, a well-balanced signal of good quality can be reproduced at an even lower coding rate.
In one embodiment of the present invention, if the scene change has been detected, the quantization parameter is preferably determined based on an assigned coefficient, corresponding to a complexity of coding the signal, and a number of bits generated and calculated during a predetermined interval after the scene change.
In another embodiment of the present invention, the assigned coefficient is preferably obtained based on an average of quantization parameters during a predetermined interval before the scene change is detected and an average of numbers of bits generated during the predetermined interval.
In still another embodiment, unless the scene change is detected, an accumulation of errors, each representing a difference between a number of bits generated and a target number of bits, is preferably obtained, and the quantization parameter is preferably determined by a feedback control using the accumulation.
In still another embodiment, a characteristic of the feedback control is preferably modified based on the errors.
In such an embodiment, the average rate can be controlled based on the complexity of coding an individual scene. Accordingly, it is possible to prevent the image quality of a scene with a low complexity of coding, in particular, from being deteriorated. In addition, the oscillation of the number of bits generated and quantization parameter can be suppressed during the feedback control.
In an alternate embodiment, a time passed since coding processing was started is preferably clocked, and the characteristic of the feedback control is preferably modified based on the passed time clocked.
In such an embodiment, the range of the variable coding rate may be broadened by performing the feedback control gently at the beginning of coding. As a result, the coding processing can be performed at a rate better suited to the individual scene. And if the feedback control is gradually enhanced along with the progress of coding processing, the average coding rate gets closer to the target coding rate, and the resulting error can be smaller when the coding processing is finished.
In still another embodiment, if the scene change has been detected, a distortion sensitivity of the newly appearing video after the scene change is preferably detected, and the quantization parameter is preferably determined based on the distortion sensitivity detected.
In such an embodiment, if the quantization parameter is decreased for a scene with a high distortion sensitivity and increased for a scene with a low distortion sensitivity, then the number of bits generated can be suppressed while making the overall image quality more uniform. As a result, a signal of quality can be reproduced at a low coding rate.
In still another embodiment, if the scene change has been detected, a distortion sensitivity of the video after the scene change is preferably detected. A first quantization parameter is preferably determined based on the distortion sensitivity detected. A number of bits generated is preferably calculated during a predetermined interval after the scene change, and a second quantization parameter is preferably determined based on the number of bits generated and calculated.
In still another embodiment, the coding method preferably further includes the step of estimating a storable capacity of a storage medium while recording coded data on the storage medium. And the quantization parameter is preferably determined considering the storable capacity estimated as well.
Another method according to the present invention is adapted to code a signal by quantization processing. The method includes the steps of: determining a quantization parameter based on an assigned coefficient, corresponding to a complexity of coding the signal, and a number of bits generated; and performing the quantization processing using the quantization parameter determined. The assigned coefficient is obtained based on an average of quantization parameters during a predetermined interval and an average of numbers of bits generated during the interval.
Still another method according to the present invention is adapted to code a signal by quantization processing. The method includes the steps of: obtaining an accumulation of errors, each representing a difference between a number of bits generated and a target number of bits; determining a quantization parameter by a feedback control using the accumulation; and performing the quantization processing using the quantization parameter determined. A characteristic of the feedback control is modified based on the errors.
Still another method according to the present invention is adapted to code a signal by quantization processing. The method includes the steps of: obtaining an accumulation of errors, each representing a difference between a number of bits generated and a target number of bits; determining a quantization parameter by a feedback control using the accumulation; and performing the quantization processing using the quantization parameter determined. A time passed since coding processing was started is clocked, and a characteristic of the feedback control is modified based on the passed time clocked.
Still another method according to the present invention is adapted to code a video signal by quantization processing. The method includes the steps of: detecting a distortion sensitivity of a video; determining a quantization parameter based on the distortion sensitivity detected; and performing the quantization processing using the quantization parameter determined.
A coder according to the present invention is adapted to code a signal representing a video or being accompanied with a video. The coder includes: a quantizer for quantizing the signal; an encoder for encoding the output of the quantizer and outputting coded data; means for supplying a quantization parameter to the quantizer; and means for detecting a scene change in the video. If the detecting means has detected the scene change, the supplying means supplies a quantization parameter, which has been determined adaptively to a scene newly appearing after the scene change, to the quantizer. Unless the detecting means detects the scene change, the supplying means supplies a quantization parameter, which has been determined at such a value as to stabilize a coding rate, to the quantizer.
According to the present invention, the quantization parameter supplying means supplies a quantization parameter at such a value as to stabilize a coding rate to the quantizer within the same scene. On the other hand, if scenes have been changed, the quantization parameter, which has been determined adaptively to the newly appearing scene, is supplied to the quantizer. Thus, even if a continuously input signal should be coded in real time, coding processing can be performed adaptively to the individual scenes, while controlling an average coding rate at the same time. As a result, a well-balanced signal of good quality can be reproduced at an even lower coding rate.
In one embodiment of the present invention, the coder preferably further includes a bit number calculator for calculating a number of bits of the coded data. When the detecting means detects the scene change, the supplying means preferably determines the quantization parameter based on an assigned coefficient, corresponding to a complexity of coding the signal, and the number of bits calculated by the calculator during a predetermined interval after the scene change.
In another embodiment of the present invention, the supplying means preferably obtains the assigned coefficient based on an average of quantization parameters during a predetermined interval before the scene change is detected and an average of numbers of bits generated during the predetermined interval.
In still another embodiment, unless the detecting means detects the scene change, the supplying means preferably obtains an accumulation of errors, each representing a difference between a number of bits generated and a target number of bits, and preferably determines the quantization parameter by a feedback control using the accumulation.
In still another embodiment, the supplying means preferably modifies a characteristic of the feedback control based on the errors.
In an alternate embodiment, the supplying means preferably clocks a time passed since coding processing was started, and preferably modifies the characteristic of the feedback control based on the passed time clocked.
In still another embodiment, when the detecting means detects the scene change, the supplying means preferably detects a distortion sensitivity of the video after the scene change, and preferably determines the quantization parameter based on the distortion sensitivity detected.
In still another embodiment, the coder preferably further includes a bit number calculator for calculating a number of bits of the coded data. When the detecting means detects the scene change, the supplying means preferably detects a distortion sensitivity of the video after the scene change, and preferably determines a first quantization parameter based on the distortion sensitivity detected. And after a predetermined interval has passed, the detecting means preferably determines a second quantization parameter based on the number of bits calculated by the bit number calculator.
A recorder according to the present invention is adapted to record a signal, representing a video or being accompanied with a video, on a storage medium. The recorder includes the coder of the present invention, and records the coded data, which has been output from the coder, on the storage medium.
According to the present invention, coded data, obtained by controlling the average coding rate and performing the coding processing adaptively to the individual scenes at the same time, is recorded on the storage medium. As a result, a signal of good quality can be reproduced at an even lower coding rate.
In one embodiment of the present invention, the recorder preferably further includes means for estimating a storable capacity of the storage medium. The means for supplying a quantization parameter of the coder preferably determines the quantization parameter considering the storable capacity estimated by the estimating means as well.
In such an embodiment, the quantization parameter for coding processing is determined based on the storable capacity of the storage medium. Accordingly, it is possible to record a signal on a storage medium with a limited storable capacity.
Another coder for coding a signal according to the present invention includes: a quantizer for quantizing the signal; an encoder for encoding the output of the quantizer and outputting coded data; a bit number calculator for calculating the number of bits of the coded data; and means for supplying a quantization parameter to the quantizer. The supplying means determines the quantization parameter based on an assigned coefficient, corresponding to a complexity of coding the signal, and the number of bits calculated by the bit number calculator, and obtains the assigned coefficient based on an average of quantization parameters during a predetermined interval and an average of numbers of bits generated during the interval.
Still another coder for coding a signal according to the present invention includes: a quantizer for quantizing the signal; an encoder for encoding the output of the quantizer and outputting coded data; a bit number calculator for calculating the number of bits of the coded data; and means for supplying a quantization parameter to the quantizer. The supplying means obtains an accumulation of errors, each representing a difference between a number of bits calculated by the bit number calculator and a target number of bits, determines the quantization parameter by a feedback control using the accumulation, and modifies a characteristic of the feedback control based on the errors.
Still another coder for coding a signal according to the present invention includes: a quantizer for quantizing the signal; an encoder for encoding the output of the quantizer and outputting coded data; a bit number calculator for calculating the number of bits of the coded data; and means for supplying a quantization parameter to the quantizer. The supplying means obtains an accumulation of errors, each representing a difference between a number of bits calculated by the bit number calculator and a target number of bits, determines the quantization parameter by a feedback control using the accumulation, clocks a time passed since coding processing was started, and modifies a characteristic of the feedback control based on the passed time clocked.
A video signal coder according to the present invention includes: a quantizer for quantizing the video signal; an encoder for encoding the output of the quantizer and outputting coded data; and means for supplying a quantization parameter to the quantizer. The supplying means detects a distortion sensitivity of a video, and determines the quantization parameter based on the distortion sensitivity detected.